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Abstract 



We discuss the flavor-changing neutral currents (FCNC) decays of the top 
quark t — * cViVj (Vi = 7, Z, g) in the framework of the Standard Model (SM) 
and in a two-Higgs doublet model (2HDM) with tree-level FCNC couplings. 
While in the SM the expected branching ratios are extremely small, in the 
2HDM they may be sizable, of order 10 -4 — 10 -5 , and thus accessible at the 
CERN Large Hadron Collider. We conclude with the interesting observation 
that the FCNC decay modes t — ► cViVj may not be equally suppressed as 
their corresponding decays t — > cVi in this 2HDM. 
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Rare decay modes of the top quark have received considerable attention recently because 
they might be a source of possible new physics effects to be searched at future colliders. 
Since the top quark has a mass of about 175 GeV PJ, the following rare processes may be 
kinematically allowed: t -»• cj @, t -> cg,cZ @, t -► bW + Z ||, t -> ciy+W" ||g], and 
£ — > cZZ 0. Even though Standard Model (SM) predicts the branching fractions for these 
rare decays to be unobservably small, it is believed that the search for large signatures from 
flavor-changing neutral currents (FCNC) involving the top quark may serve as a unique test 
of the SM |5|JT8[1. The aim of this communication is to point out that the FCNC transitions 



t — > C77, t — > cjZ and t — > egg also provide a detailed test of the SM and some of its 
extensions, e. g., the two-Higgs doublet model (2HDM). 

Just as it is the case with the FCNC one-vector boson decays t — > cVi (Vi = 7,(7, Z) 
|2|||, the two- vector boson decay modes t — > cViVj are expected to be negligible in the 
SM because they are induced at one-loop level and in addition the Glashow-Illiopoulos- 
Maiani (GIM) mechanism suppresses effectively FCNC transitions involving virtual down- 
type quarks. However, in the most general version of the 2HDM (known as Model III), 
where quarks are allowed to couple simultaneously to more than one scalar doublet J7| , it is 
possible that important effects will emerge in scalar FCNC couplings involving quarks of the 
second and third generations. Unlike earlier versions of the 2HDM, in Model III no ad hoc 
symmetries are invoked to eliminate tree-level scalar FCNC couplings, but instead a more 
realistic pattern for the Yukawa matrices is imposed and constraints on the scalar FCNC 
are derived from phenomenology ||. The tree- level scalar FCNC interactions are given by 

£y,fcnc = Cij sin afifjh + f y cos afifjH + £ y cos afrffjA + h.c, (1) 

where we are using the Higgs mass-eigenstate basis with the light and heavy CP-even Higgs 
bosons h° and H°, respectively, and the CP-odd Higgs boson A , a denotes the mixing 
angle, and £y corresponds to the off-diagonal Yukawa couplings. It is convenient to use the 
parametrization introduced in ||: ^ = ^ m j mj ; where the mass factor gives the strength of 
the interaction while the dimensionless parameters are fixed by low energy experiments. 
Although there are strong bounds on couplings involving light quarks, no stringent bounds 
exist for X tc , and it is feasible a less suppressed strength for the interaction tc0°, with (fP k 
any of the three physical Higgses of Model III. 

In the SM and Model III the FCNC decays t — ► cViVj may proceed through the resonant 
diagram shown in Fig. 1, where (fP k * represents the exchanged Higgs, and the dots denote 
the couplings of this scalar with the particles in the initial and final states. The amplitude 
for this diagram can be written in general as 

M (t^ = £^(s) M(t^ c<p° k *) M(4 * - ViV s ), (2) 

k 

where P^o{s) is the resonant propagator of the exchanged Higgs, which transfers a squared 
4-momentum s = (k\ + k 2 ) 2 , which in turn is restricted by kinematics to lie in the range 
(rriVi + my ) < yfs < (m t — m c ). The partial amplitudes in Eq. (0) can be expressed in 
terms of the (fP k couplings 

M(t^c<p k *) = u( q )T tC(f> ov(p), (3) 
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m(4 * -> viy,) = irfl e^h, \ 1 y(k 2 , a 2 ), (4) 

where ^tc<f>° k characterizes the coupling tc<fi° k , which in the SM arises at one-loop level but 

in Model III it is a tree-level effect, whereas characterizes the one-loop level coupling 

4>{ViVj |nj. In the case of the channel t — > egg, Eq. ([|) must be modified to account for 
the SU(3) gauge structure of gluons. The decay width T(t — > cVjVj) can be obtained in the 
usual way and it is given by 



r(t -> cViVj) = occ _ 3 „, / n A(m i ,m c ,Vs)A(Vs,m F „m l A) \M(t -> cT^V^)| 2 <is, (5) 



'(mt-mc) 2 
2567T 3 m t i(m y .+m v ) 2 

where X(x,y,z) is a phase space factor. 

To realize how the top quark decays t — *> cViVj get enhanced in Model III, it is necessary 
to examine the branching ratios in the SM, where only the FCNC two-vector boson decay 
t — > cW + W~ occurs at tree-level with a branching ratio of order 10~ 14 — 1CT 12 ||. As far as 
the decay modes t — > cryj, t — > cjZ and t — > egg are concerned, in addition to the one-loop 
Feynman graphs, the resonant diagram shown in Fig. 1 contributes to the decay amplitude. 
In this case the SM Higgs <^g M is the mediator and the coupling ic<^g M is induced at one- 
loop level. The contribution of this diagram seems to be of order g 6 , but a careful analysis 
shows that on-resonance it becomes of order g 4 , being the dominant contribution for light 
bosons and competing with the one- loop contributions [11|. The SM coupling tc<p^ M can be 
expressed as 

r **g M = E V tk VC c (A L (s,m 2 k )P L + A R (s,ml)P R ), (6) 

k=d,s,b 

where Pl and P R are, respectively, the left- and right-handed chirality projectors, whereas 
Al and Ar are given in terms of scalar integrals. We have calculated r tC( ^o and our result 



nj agrees with the one presented in JTJj and later reproduced in [pT3| , [T4|| for the SM decay 
o 

SM- 



t — > c0g M . This coupling has also been studied in the Minimal Supersymmetric Standard 



Model ||15|| . In the zero charm quark mass limit the function Ar vanishes and can be 
neglected in the calculation. From Eqs. (@)-(§) it is straightforward to obtain, after dividing 
by the main top quark width T(t — > bW), the SM values B(t — > C77) ~ 10 -15 - 10 -16 , 
B(t -> c-fZ) ~ 10" 15 - 10^ 16 , and B(t -> egg) ~ 10" 14 - 10" 15 for m z < m^o < m t . 

In the framework of Model III we have studied the decays t — > C77, t — > cyZ, t — > egg, 
and for completeness also the channel t —>■ cW~W + , for which we agree with the results 
found in . We will not discuss the mode t — ► cZZ because it is highly restricted by phase 
space. Since the most relevant phenomenological tree-level FCNC effects are due to the 
light scalar h° M, the heavy ones H° and ^4° are unimportant for our present discussion. 
In fact one can assign to them an arbitrary large mass, namely 750 TeV. The same value 
was assigned to the charged Higgs mass m#±, which enters in the expressions for the one- 
loop induced coupling (fyViVj, though we found that the results for the decay widths are not 
sensitive to this choice. For simplicity we have chosen X tc to be real, thus the branching ratios 
obtained from Eq. (|5|), after dividing by T(t — > bW), turn out to be proportional to the factor 
fx — AjcSinacoso;. Plots for the branching ratios scaled by the factor fx are shown in Fig. 
2 as a function of the Higgs mass m^o. This graph teaches us that the branching ratios reach 
their most significant enhancement in the resonance region (my. +my.) < m^o < (m t — m c ) 
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where the channels t — > C77 and i — > C7Z may have branching fractions as high as 10 -5 - 
10~ 4 , while those of the modes t — > cW + W~ and t — > egg may reach values up to 10~ 4 . The 
enhancement is due to the combined effect of the tree-level FCNC and the resonance of the 
Higgs boson. On- resonance, the decay t — > cViVj proceeds in two stages: first the top quark 
decays as t — > ch°, with the on-shell Higgs decaying then as h° —>■ ViVj. This contrasts with 
the pure three-body decay that occurs outside the resonance region. Although in Model III 
the decay t — > cW + W~ is induced at tree-level, it is highly suppressed by phase space, and 
its width is only one order of magnitude greater those for the decays t — > C77 and t — > C7Z. 

The possibility of detecting FCNC top quark transitions at the Tevatron has been previ- 
ously investigated, for instance through the reaction c + g — > t + A° ||16|| . Here we will discuss 
the mode t — > c + 77 as a representative case of new signatures associated with FCNC top 
quark decays. This reaction turn out to be very clean when it is attempted to separate 
the signal from background. Let us consider the situation at the Tevatron, where it will be 
possible a production of top quark pairs through quark and gluon fusion with a cross-section 
of 5 x 10 3 fb ||17|| . If the top quark decays into c + 77 with a branching ratio of order 10 _fl 



an integrated luminosity of 10 2 fb^ 1 would be needed in order to produce at least one event, 
which exceeds the planned luminosity (of order 1 fb -1 ). Then it seems hopeless to detect the 
FCNC two-photon top quark decay at the Tevatron. Nevertheless it remains the possibility 
of detecting the Higgs decay A —>■ tc, which may have a branching ratio of order unity for 
m^o > m t + m c . At the Tevatron the cross-section for the reaction gg —>■ A —>■ tc is about 
10 3 fb, and the same is true for the cross-section of the main background, which is expected 
to come from the charged-current reaction q + b — > c + t. Thus it seems feasible that an 
adequate implementation of cuts will allow to identify the signal. 

On the other hand, the situation looks more promising at the CERN Large Hadron 
Collider (LHC), where the cross-section to produce top quark pairs will be 4.3 x 10 5 fb, and 
a branching ratio of order 10~ 5 will reduce this cross-section down to 4.3 fb. We will use a 
conservative efficiency (0.05) for the production of the t + c + 7 + 7 events at the LHC. We 



obtained this efficiency from the detailed study of T. Han et al. [jn| on the single top quark 
production via FCNC couplings at hadron colliders. With an integrated luminosity of 100 
fb~l we then get only 21 £ + + 7 + 7 events per year at the LHC. The main background 
to this signal comes from the production of g& ^£ + + 7 + 7 and qq — > £ + f> + 7 + 7 
events, where the final b may be misidentified as c. The analysis in [|18j for the single top 



quark production can be used also to show that, after the inclusion of the suppression factor 
(|f ) 2 to account for the two emitted photons, these background processes will have less than 
one event at the LHC and thus will make feasible the detection of our signal. Even more, 
our t + c + 7 + 7 signal is so distinctive that its detection should be difficult to miss at 
the LHC: two highly monochromatic photons with an invariant mass peaked at m^o plus a 
single quark and a jet. A more systematic study of all possible backgrounds is, of course, 
beyond the scope of the present rapid communication. 

Let us consider another implications of our results. If we assume X tc to be of order 
unity, the branching ratios for t — > cViVj may reach values up to 10~ 4 in the 2HDM, while 
the branching ratios for the decays t cVi (Vi = 7, Z, g) could be of order 10~ 5 — 10~ n , 
depending on the Higgs boson masses and the value of tan/3 = with u 12 the vacuum 
expectation values of the two Higgs doublets |L3|. Thus, we find that for a light Higgs the 
three-body decay modes t — > cViVj may not be equally suppressed as the two-body decays 



4 



t — > cVi in the 2HDM. Our result represents an explicit realization of the scenario previously 
conjectured in studies of the rare processes b — > 577 fl9 |, fi — > e77 [2(J and v — > z/77 



where the corresponding branching ratios get enhanced in comparison with those of 
the one-photon processes. 

A large two-photon FCNC transition may also have implications for the decay B° s — > 77, 
which can occur through an annihilation graph mediated by the Higgs boson A . A simple 
PCAC evaluation shows that the corresponding branching ratio is of order 10 -7 , which is 
comparable to the SM result. In order to obtain further conclusions one needs to include 
QCD corrections, as has been done recently for the SM and virtual contributions of the 
charged Higgs [p2]| . 

To close this communication we would like to stress the relevance of our results. Even 
though the SM is a well established theory supported by a plethora of experimental data, it 
leaves us with some unanswered questions. In particular, the scalar sector with its elusive 
Higgs boson remains as the SM most puzzling piece. Supersymmetric Grand Unified Theo- 



ries allow for a relatively light Higgs boson in the range 100 — 200 GeV ||23|| , and it is possible 
that in the next years it will occur the milestone discovery of a light Higgs boson. Mean- 
while there is an incessant search for any evidence of new physics that could enlighten the 
road to a more comprehensive theory of elementary interactions. In this context, the results 
presented in this rapid communication open up the possibility of looking for signals from 
physics beyond the SM in the FCNC top quark decays t — > C77, t — > cjZ, t — > cW + W~, 
and t — > egg. 
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FIGURES 
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FIG. 1. The decay t — ► cV^Vj in the SM and Model III. The dots denote couplings that can be 
induced at tree or one-loop level. 
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FIG. 2. Scaled branching ratios for t — > cV{Vj in Model III: t — > C77 (solid line), i — > C7Z 
(points), i — ► cW + W~ (hollow circles), and t — > egg (full circles). A value of 750 TeV is used for 
the masses m A o , m H o and m H ± . 
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